Quantified acoustic–optical speech signal incongruity identifies cortical sites of audiovisual speech processing

Quantified acoustic-optical speech signal incongruity identifies cortical sites of audiovisual speech processing.

A fundamental question about human perception is how the speech perceiving brain combines auditory and visual phonetic stimulus information. We assumed that perceivers learn the normal relationship between acoustic and optical signals. We hypothesized that when the normal relationship is perturbed by mismatching the acoustic and optical signals, cortical areas responsible for audiovisual stimul...

Audiovisual speech processing in visual speech noise

When the talker’s face (visual speech) can be seen, speech perception is both facilitated (for congruent visual speech) and interfered with (for incongruent visual speech). The current study investigated whether the degree of these visual speech effects was affected by the presence of an additional irrelevant talking face. In the experiment, auditory speech targets (vCv syllables) were presente...

Audiovisual processing of Lombard speech

Perception results are presented that address the role of Lombard speech in auditory and audiovisual speech perception. Basically, visual enhancement neutralizes the advantage of Lombard speech observed for auditory perception. It remains an open question whether or not Lombard speech is preferable for perception studies of speech in noise.

Audiovisual Integration in Speech Processing

Perceptual processing of audiovisual Lombard speech

Seeing the talker improves the intelligibility of speech degraded by noise (a visual speech enhancement effect). This experiment examined whether this enhancement is greater when the speech signals were recorded in noise compared to when they were recorded in quiet. Ten sentences were spoken by four people either in-quiet or whilst they were listening to cocktail party noise (in-noise). The vis...